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ABSTRACT 

A study compared the nature and frequency of faults 
and errors in reference list entries and direct quotes selected from 
73 articles in 8 issues of the "American Educational Research 
Journal" (AERJ) with authors sampled from 10 previously studied 
journals. All departures from the original (additions, omissions, or 
changes) were labeled as either a word or punctuation deviation. Of 
856 AERJ reference list entries that were retrieved, 46.26% contained 
at least one fault or error, which is similar to the fault/error rate 
of the previously reported data. The most frequent discrepancy was 
omission of the author's middle initial, with the second being a 
punctuation fault. Other errors and faults included: (1) misspelling 
of author surnames; (2) erroneous volume numbers; (3) erroneous page 
numbers; (4) word changes, additions, replacements, and/or omissions 
in the title of the article being cited; (5) quotation marks used for 
paraphrases; and (6) direct quotes presented as paraphrases. The data 
compiled here, added to the findings from previous studies, leave 
little doubt, although the samplings are relatively small in light of 
the large numbers cf authors, that faults and errors in scholarly 
writing are widespread, and almost certainly increasing in frequency 
of perpetration. (One table of data is included and 26 references are 
attached.) (PRA) 



******************************************* 

* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



iH 
00 

o 

CO 



Direct Quote and Reference List Entry Faults and Errors in a 
Sample of Articles from the American Education al Research 
Journal. Compared with Findings from Previous Research 



Arden White and Nelda R. Hernandez 
University of Wyoming 



1991 



If 



ERIC 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



0 8 DEPARTMENT OF EDUCATION 

EDUCATONAL^U^NFORMATWN 

°" a,na, ' ng 'l W en marie H mm— 

C M<"Of changes have 

reproduction Quality ^ 

" T^ rt n»oajed mihisdocu 



BEST COPY AVAILABLE 

2 



AERJ Study 
2 

Direct Quote and Referent List Entry Faults and Errors in a 
Sample of Articles from the American Educationa l Research 
Journal. Compared with Findings from Previous Research 

Background 

Faults and errors in reference list entries in medical-related 
journals have been documented over the past decade in several 
reports (Eichorn & Yankauer, 1987, Goodrich & Roland, 1977, 
Hartmann, 1984, Key & Roland, 1977; Poyer, 1979). Faults and 
errors in quotations, as well as references, were reported in two 
studies (de Lacey, Record, & Wade, 1985, Lowry, 1935). Similar 
studies of social sciences journal article reference lists (Boyce & 
Banning, 1979, A. White, 1987a, 1987b, 1987c) and direct quotes 
(Hernandez & White, 1989) also have been been reported. Sabine 
(1985) reported on error rates and published corrections for 10 
biological sciences journals. Several letters have appeared in 
journals, in which the writers have expressed concerns about 
articles they have judged to contain overgeneralizations from data 
(Borit, 1982, Koshland, 1987, Rifkin, Quitkin, & Klein, 1979, 
Schwartz, 1970, R. R White, 1987), changes of wording (Ryan, 
1975), and changes in meaning due to quoting out of context 
(Chamberlim 1980, Young, 1982). The accuracy of interpretations of 
content in scholarly literature also has been questioned (Eichorn & 
Yankauer, 1987), and in the lay press reporting of medical news 
(Proudfoot & Proudfoot, 1981). 
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The general tenor of findings from these studies is that faults 
and errors are widespread. The definition of faults for reference list 
entries includes deviations from the original that do not inhibit or 
preclude retrieval of the original source. Errors, on the other hand, 
include those changes, such as misspelled first author surnames, or 
incorrect years, volume numbers, or beginning page numbers, that 
cause varying degrees of difficulty in retrieval. 

Deviations in direct quotes— which readers are given to assume 
must be exact reproductions of the original [disregarding possibilities 
of quoting out of context] — also vary in seriousness. Many of the 
departures from the original, labeled faults, are not disruptive to 
the conveyance of meaning. Others, labeled errors, change the 
meaning, ranging from a small change to a complete reversal. 
Rationale and Pmr.c>d\)rc>* 

Given the studies already done, why do another? One argument 
for doing so is that authors of articles from only a small percentage 
of all journals have been studied. Any sample which is small in 
proportion to a population could be quite unrepresentative. 

The study reported here extends findings from previous studies. 
Seventy-eight articles in eight issues of the American Friuration al 
Research Journal [ 22 (3-4), 21 (1-4), 24 (1-2)] were studied. 
Reference list entries referring to articles appearing in journals 
owned by the authors' home university library were compared 
visually with the originals. In like manner, direct quotes, or 
statements contained in double quotation marks that appeared to 
be direct quotations, whether in journals or Looks, were sought. All 
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of those that could be located were compared word by word, and 
for punctuation, with the original source. 

Procedures were identical to those used in the recently reported 
studies by A. White (1987a, 1987b, 1987c) and, for quotes, by 
Hernandez and White (1989). In brief, a direct photocopy of the 
quote or reference list entry was compared with the original. If no 
fault or error was located, the quote or reference list entry was 
tallied as accurate. Those with any deviation were tallied as 
incorrect. In addition, specific tallies were made of the various 
faults or errors. These data for reference list entries are displayed 
in Table 1, along with data from the previous studies by A. White 
cited above. 



Insert Table 1 About Here 



Findings for Uoforpncp !.kt Fntri<^ 

Of 856 AERJ reference list entries that were retrieved, 396 
'46.26%) contained at least one fault or error. This overall 
"fault/error rate" is quite similar to those for the authors sampled 
from the 10 journals reported in A. White's previous studies. The 
median fault/error rate for these previous data is 42.15. 

Counting all faults and errors, 609 were located in the entries 
that were not wholly identical to the original. Obviously, some 
entries had multiple faults and/or errors. Even so, the omission of 
a several-word subtitle, for example was counted as just one error, 
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as the omission of a comma was counted as one punctuation fault. 
Had all word omissions and additions in article titles been added, 
the error figure would have been much higher in these data fields. 
The single most frequently committed discrepancy (n=191; 31.4% of 
609) was omission of the middle initial of article authors. The 
second most frequent discrepancy was a punctuation fault (n=72; 
11.8%). Neither of these fault/error commissions is especially 
misleading or harmful, in most instances, although an author who 
uses the middle given name often will be mis-identified when the 
middle initial is entirely omitted. Even then, retrieval is unaffected 
unless the first initial is in error. However, misspelling of the 
surname of the first author can be a bigger difficulty. 

Of the total 856 reference list entries checked, 13 (1.558) of the 
first author surnames were misspelled. This figure is not quite 
twice that of the year being wrong in the listed reference (8 of 856, 
0.93%). However, volume number was in error in 22 instances 
(2.57%). This finding parallels percentages in the previous studies by 
A. White, cited above. Just why volume number would be in error 
more frequently than year is not readily explainable. The same 
uncertainty rxists for page number errors. In this study, beginning 
page number was found to be in error 10 times (1.17%) while end 
page number was wrong a little more than three times as often 
(n-34, 3.97%). Both page numbers were wrong in 11 instances, as 
well. 

Among the more interesting of departures from the original 
entries were word changes, additions, replacements and/or 
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omissions in the title of the article being cited. Although these 
discrepancies have nothing to do directly with retrievability, they 
may have quite an influence on a researcher's decision about the 
relevance of the article in planning research. A suitable work may 
be by-passed if an incorrect title serves as a distortion of the article 
contents. Some incorrect entries had combinations of these^errors, 
making the distortions even more extreme than a "simple" 
omission, or replacement, or addition. 

Some examples of title changes may illustrate the various 
distortions that occur. Omission of the three-word sub-title "Some 
behavioral data" changed an entry enough so that an unwary 
reader might infer that the article is a commentary or theoretical 
piece, instead of a report of empirical research. Even the one-word 
omission of "emotional" from the two-word term "social-emotional" 
could influence some readers not to seek the original source. 
Another omission, "pupil behavior in, " resulted in a substantial 
change in the correct title: "Stability of pupil behavior in 
short-term classroom observations." In another title distortion, the 
words 'appropriate' and 'inappropriate' somehow got substituted for 
'stereotyped' and 'nonstereotyped, ' linked in both instances with 
the word sex. In another instance the one-term major title 
omission of "Self-concept" left the title as "Validation of construct 
interpretations, " and the reader with a surprise in the offing if the 
original article were to be retrieved. 

In some instances the citation of a reference within the text of 
an article may be sufficient to indicate for the reader the likely 
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value of the source for that reader's purpose. But the brevity of 
writing encouraged or required by many journals leads to citations 
that depend on reading reference list entries for making the 
judgment about whether or not to retrieve the original source. An 
error in the title in the reference list entry then can be pivotal in 
the reader's decision. For example, the one-word omission 
(underlined here) "Listening to stories may change children's social 
attitudes" leaves the reader with the impression that the article is 
much broader than had the word been included. The writer that 
replaced silpnoe with soul in the brief title "Science, silence, and 
sanctions" changed the apparent content of the article profoundly. 
In another instance, when Catholic schools somehow got changed to 
private schools, the implied content of the article is distorted 
considerably. 

Findings from the Quotations 

An unexpected difficulty infused verification in 39 articles in 
AERJ that appeared to have one or more direct quotations. 
Quotation marks suggestive of a direct quote were used in 17 
instances that probably were paraphrases. In another 24 instances, 
efforts either to locate a statement that appeared to be a direct 
quote, or to find any indication of paraphrasing, failed. 
Additionally, 34 statements that were located and found to be 
actual direct quotations had no page number cited. Three were 
located that had a wrong page number cited. Overall, 95 
statements were located that were found to be direct quotations. Of 
this total, 61 (64.2%) were exact duplications of the original. This 
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percentage is somewhat higher than Hernandez and White (1989) 
found in that study. However, the ambiguity here is that 34 
quotations had no page number, certainly an error by the 
author (s), and 24 apparent quotes were not found, 17 of which 
probably were paraphrases in quotation mark— a questionable 
practice in style and form, at best. 

Of the located faults and errors, as already noted, absence of a 
page number was most frequent, but was confounded with the 
(apparent) paraphrasing within quotation marks. Next most 
frequent (N=14) were omissions of one or more words, followed by 
replacement of one or more words (N = 12). Seven instances were 
found of a word or words being added to the original quotation. 
Only 14 instances of punctuation discrepancies were found. This 
pattern of wording deviations in aggregate being about as frequent 
as all other faults and errors combined is similar to the findings in 
the previous study, when the data are adjusted for the apparent 
paraphrasing without page number citation. All together, these 
findings about quotations suggest that their reliability is far from 
perfect. The serious reader must retrieve the original source and 
read it carefully, if usage of the quotation is of importance to the 
work at hand. 
Discussion 

While they are pointedly illustrative and similar to those found 
in previous studies, citing these departures from the originals does 
little to suggest either how such errors occur, or how to prevent 
them or correct them prior to publication. Sabine (1985) argued 
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that most errors probably originate with the author (s), who make 
the mistakes and then do not find and correct them. Even if the 
problem is very widespread, "universal," according tc Roland (1976, 
p. 717), the primary fault almost certainly lies with the 
originators— the author(s). Although discussing more severe 
problems than those presented here, the argument by Engler, 
Covell, Friedman, Kitcher and Peters ( 1987) that peer review can 
be only a partial guard would seem to apply, just as they argue 
that editors are relatively powerless to identify and correct errors 
or faults of any but the more detectable sorts. Haworth (1985) 
suggested that editors check the less recent references, "which may 
be third or fourth hand" (p. 1282). Obviously, Haworth was also 
asserting that at least some authors do not retrieve each and every 
original source and read it before citing it in their piece. While 
discussing development of the footnote as a social device for dealing 
with ownership of intellectual property, Kaplan (1965) also 
speculated about writers' lack of care in reading of original sources. 

The data compiled here, added to the findings from previous 
studies, leave little doubt, even though the samplings are relatively 
very small in light of the very large numbers of authors, that 
faults and errors in scholarly writing are widespread, and almost 
certainly not decreasing in frequencies of perpetration. We 
speculate that, if anything, they very likely are increasing. 
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Table 1 

Numbers and percents of faults and errors in reference list entries 
from 35 volumes of 10 journals previa ;$ly studied and 78 author sets 
from 2 volumes of the American Educational Research Journal. 
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